The present invention relates to digital image processing error concealment, and in particular, to occlusion handling for motion compensated interpolated frames.
In a digital image processing system, images are represented as binary data (discrete values represented as zeros and ones). Changing images, such as video or graphics, are typically broken down into a stream of frames. Each frame may include a number of static objects represented by an array of pixels. Different values for different pixels correspond to a wide range of colors (e.g., RGB or YUV) across each frame to represent different visual objects in the frame. In a video or graphics system, for example, the objects may change location from frame to frame, thereby creating the appearance of movement.
FIG. 1 illustrates an image 101 with a moving object 102. Object 102 may move across the image with a velocity, v, which specifies both the direction and speed of the object's movement across the image. As objects move across an image from frame to frame, leading edges of an object may cover some background images and trailing edges of the object may uncover other background images. This is illustrated in FIG. 2, which shows the movement of an image across multiple frames. In FIG. 2, an object 102 changes location between frames 201A and 201B. In this example, object 102 moves horizontally a distance, d, between frames. As illustrated in frame 201C, a first region 211 is occluded by the leading (or forward) edge of the object, and a second region 210 is revealed by the trailing (or rear) edge of the object.
The occlusion of background images by the leading edge of a moving object and the revealing of background images by the trailing edge of a moving object often results in error regions in the images and can impact image quality. For example, a typical artifact that is associated with the occlusion is a “haloing” artifact around object boundaries. There are at least two causes of such an artifact. First, errors may result from a coarse motion grid. Typical motion estimation algorithms do not estimate motion at pixel grid level. Rather, the typical resolutions of the motion estimation algorithms are either 8×8 pixels or 4×4 pixels, for example. This coarse resolution causes an inadvertent discontinuity around boundaries. Second, motion estimators may not be able to reliably measure around object boundaries using information from two input frames. The useful information may only be present in one of the frames, and thus, block based motion estimation techniques may result in large errors around the boundaries and estimate incorrect motion. The area of ambiguity may be dependent on the motion between the foreground and the background images as well as the temporal position of the interpolated frame.